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' EFFECTS OF ORDER FOR THE ADMIlflSTRATIOH 

, ^ . ^ 

OF SUBTESTS Oh STAHOffRDIZED READIfte TEST SCORES • 

* 'is' 

■ Many standardized reading tests provide a total test score 
cdmposed of scores yielded by various subtests. Subtests are 
designed to' mea§jjre specific skills which have been judged to be 
integral to* the reading procesSs. -The value of scores frpm sub- 
test^ has been questioned by Farr (1969)^s to whether the sub- 
tests have sufficient discriminant validity to measure separate 
skills, but 'test publishers continue to develop subtests an^/ ' 
often provide norms for separate subtests. . ^* 

Research in readino p€r4()dically7o£i5?es||^n sefwrate skill 
'areas and is concerned with only selected aspects of a total 
standardized test score. Efficient time usage vwuTd seem to 
sugqest'that jf interest is in a specific skill area, only that 
portion of a test dealirtg with the specific skill be administered 
This wuld seem espec^ially appropriate /in a subskill such as com- 
prehension, which has be^n identified /as a separate skW (Carrcfl'j, 

1972*). • ■ ' . * '/ 

A recent s,t4xly by Peterson et a(l.Ml976) is an example of 
research in which a subtest score from a standardized reading jtest 
was used. -^Th.is -study was an ^tteispt to determine -whether cau- 
tiousness or original ^hinkip-^ personality traits miaht influence 
cloze pcirformance independent^of reading ability. To achieve 
cjoz6 perfonnance4mle^)^ndent of reading ability, subjects were 
^fliv^ii cloze passages matched to their reading comprehension levels.. 
Reading comj^h«nSion levels i*ere determined by scores on the com- < 



prehension subtesfof the Nelson-Penny Reading Test . The com^ 
prehension subtest alone was administered because:. 1) administer- 

, ing one subtest 'seemed to be an expidltious use of subjects' time. 
2) a general comprehension ability is reporte^to exist somewhat 
independent of vocabulary ability (Carroll, 1972), 3), (prehension 

; subtest 'Scores and norms are provided in the Examltteif^s^Maniial 
separate from scores for the subtests of vocabulary^ and rate, and 
from total test scores, and 4) the comprehension test appeared 
quite independent of the vocabulary test in tehns o'f administra- 
tive detail is. 

Data from the Peterson et al. (1976) study indicated that- 
mean score results on the completed cloze passage exercises were 
higher than expected when cofrtpared to previous research reports 
(Bonnuth, 1967; and Peterson et al.", 1972)- The results did not 
affect the credibility of the study but did raise the question 
as to whether the cloze passages were indeed mat^:hed to the sub-, 
ject's appropriate reading level. . 

An earlier study by Peterson et al. (1972) had employed the 

•same procedure to assign the identical cloze passages to subjects - 
except the entire Nelson-Denny Reading- Test was administered to 
determine readirtg level . In this study, 5^oze score results were 
within the expected ranoe.Of success as indicated by Bormuth's ' 
(1957) research. 

The maior difference between the Peterson et al. (1976) study 
and the Peterson et al. (1972) study was the manner in which the 

' reading test was a^inistered.. The cloze test resQlts front the 
two studies suggestjthat the subjects in the Peterson et al. (1976) 



\stM<Jy received clqze passages that were pslow their readina in- 
itVuctionaT Ifevel whiU the subjects in the Peterson et al. '(1972) 

-sttKly received passages 9t their, instructional Jevel. The sinale , 

' * ■ • \ " * ' 

difference in the a'ssignment procedure for cloze pass^sesj was' 

vjheth^r the subjects were'admlinlstered the entire Nelson- Denny 

Reading list or if the subjects received just -the comprehension 

subtest. " Thus, the manner ifi which the readina tests were adnjin- . 

istered seemed to be related to th^ scores on the\ individual sub- 

. tests. ^ ' . ^ ' \ 

* •» t 

Purpose • - . . . 

'The purpose of "^his study was to. systematically examine the 
^ effects of order*for the administration of subtests on- a standard- 
ized readiria tist. The" answer to the Vollov/inq question was spuaht 

^ What difference in subtest scores and total test score will^ 

• be found when the order for the administration of the sub- 
tests for vocabulary and comprehension are reversed?/ 

. - ! ' . 

Subjects , , 

' A total of 131 subjects in grades ten, eleven andl twelve in 
North Dakota and Hyoming pStrticipated in thfe study. Sixty-niner 
subjects resided in North Dakota and sixty-two subjects lived ?n 
Wyomihg. All subjects were residents of communities^wlth a pop- 
ulation^of 25,000 or less. - - 

r 

^ Procedure 

Material s :l The standardized test used for data 'collection 
was the Nelson-Denrvv Reading Test , Forms A iind D. Subjects^ ij^ 
North Dak^a were administered Form A while subjects in"V(j(pmlfn3 
received Torm D. 
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The primary question under investigation was the Effect of 
order for the administration of thfe subtests, vocabulary amfcom- 
prehension^- within the Nelson/penny Reading Test. Two forms of 
the test were u^ed to collect data and wh^iU^the format fcr the 
two forms is identical. Form* A was copyri<!hted in 1960 and Form D 
in 1973. Form A v/as u^ed excTu^ively with the subjects in Hortit 
Dakota and Form D was lised with subjects in.Vtyom^g^. 

Use of tVsts in this manner'present^ a possible confoimdinq 
of the variable test form with state. This variable will be re- 
fer^ed to as "state" even thouqh the possibility exist^s that it 
'represents effects due to the two different forms. If an inter- 
actioQ exists involvinq this 'yari able, caution must be used* in 
interpretinq the results. 

In order to test for interaction Involvinq. the state variabli 
an analysis of variance Was conducted usinq factors o/ state and 

^order of subtest ,a<imini strati on. Tables 1 and 2 preserrt d^ta 

_ - I 

relative to this analysis. \ % ^ 

✓ 

Rfesults indicate no significaat interaction between the vari 
ables state anti order. The lack of interaction suggest a consis- 
■ tant'pattern in each state of mean score results for the order ^ 

of subtest administration. As the results were in the same direc 

■ I " * - 't f. 

tion for bothstfies, it seemed. appropriate Jo pool the data • 

leaving unresblved the true nature of the' state vs. forms con- 
founding. • * ' • 

Testing . < All. testing was by experienced educators and 
test administration procedures foil wed the instructions' in the 
examiner's Manual., The only deviation from the^ manual. was the 



order 1n v/hich subtests were administered which was the Independent 

V variable' for the stiyy. ' . • 

Two orders of subtest acbnlplstra^lftP determined. Order 

one consisted of ddmlnlsWlhg th^ubtests as directed In the 

Examiner's tiapual which Instructs \et the subtest for vqcabulary 

' ' « ' 

be given first and the subtest for comprehension second (VC). * 

Order two reversed the sequence for adfninisterlng the Subtests, 

thus comprehension was given first foll^^ejAy vocabulary (CV)s - 

Subjects within each state wei;^ randomly, assigned to one of 

the order groups VC or CV-. Sub^actjs went with the examiners tO' 

a testing room where the test was adminlst.er^d to groups of ten 

, to twenty students, / ' ' 

- . The order fpr administerfng sujltests J»as_^ counter-balanced < 

for exfiralners by-state. Examiner oni^ administered the subtests ^ 

^ in the sequence* VC, CV, VC, CV. ,Exi\ji1ner two reversed the ordir 

using the sequence: CV, VC, CV7"VCr 
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Data, AnaJysiS' . ^ ^ 

Tables^ 3 Av and.B report t test results reTatlve the 



effect of order for sirt)test administration on the subtests for 
, • ■ f 

vocabulary and comprehension an^l the total test score. 



Findings 

- The central hypothesis under investigation examined the 
effects of ,th|b order .fo^ tlie administratiornTf the .subtests in 
the Nelson-Denny Reading Test on subtest and total test scores . 
Results indicated that. the order of administration had no sig- 
nif leant- effe(;t on the vocabulary subtest, nor on the total test 
score. . . ' ' . • 



the* order ofj^dmi^istration of subtests did, hoivever, have a 



significant effect on the results of the comprehension subtest. \ 
. Subjects taking the subtests Mn' the order ^instructed by the Examiner's 
V jitnual , vocabulary thep comprehension, s^red higher than^ subjects 

taMiig the comprehension subtest followed by the vocabulary subtest. 

Impljcatidns ' 

The results of. this investigation4ndicate that the or^er 
\for admtni strati on of subtests of a standardized readinq test or 
the partial administration of a reading test may influence the 
tesl" score. In the present stydy, this efft^t-was a Tewer score 

on the comprehension subtest when this test was administered prior 

• * * • . 

, , to the vocabulary subtest. ^ 

Research which employs standardized reading tests should be 

aware that raw scores may va>y depending upon whether the entire 

test is administered^ selected subtest^'of the test are used. 

The variablity of scores is particularly important if iiorm tables 

are to be used. ; . 

* In this period of accountability and critej?ion reference 

. testing, it may be temjJting t^dmiftister only ' those portions of 

tests which appear most .relevant to instruction. It should be 

♦ \ 

remembered that normS for standard,ized tests may have been rfeveloped 
by administering the entire test battery. Use of norm tables .fPr. 

• subtests when only a. portion of the test has beeij^administered 
seems questionable. ' ^ . 
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TABLE .1 . . • 

Analysis of VaHance Results 
for the Factors 
of Stat^ and Order" 
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P-Value'" 


state 
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1„ 542. 465 ' 


1,542.4*65 
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** 
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Order 
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231 .398 


.525 


.470 
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114. 43» 


114.430 - 
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Error . 


>27 
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55,959,789 


440.707 
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riean Scores and -Standard Deviations 

for the Factors 
, *• I of State and Order 



State 
* • 


Order 1 




Order 2 , 

* 






Mean Sco're 




N 


Mean Score • S.D. 


N 


Wyoming 


68.471 


24. 798 


34 


67.679. ■. 20.891 • 


28 


North Dakota 


03.457 . 


18.460 


35' 


' 58.912 19.311 
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. • . 1 . ^ ~^ TABLE 3 . • . , : 

■ >\ '.t'' v" - ' * . • * 

^ .^itplsi^sul'ts wirthe Vocabulary Subtest' 
•for the EffftMlof* Order of .Subtest Administration * ^, 
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t 2-Tail 


Order 


• N 


Mean 


c S.D. . 


SE 


Value • Probability 

— r- — — » — 


■ VC' ~> 


69 , 


'27,319 


13.533 




-.31 " .75 
















62 


28.000 


; 10.926 


1.388 

















C ^ TABLE 4. . * ' 

, ... . .-^ _ 

t Test Results on the Comjsrehen si on Subtest 
for the Effect of OV-der of Subtest Administration 
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Order 


N Hean * 


S.D. 


SE 


t- 

^ Value 


^ . 

2-Tai,l . 
Probability | 


vc 

- t • 
I • 

' cv 


69 38.454^ 
62 '34.871-.. 


' 10.018 
i-0.982 

m 




*-1.96 


C * > 

■ .05 

* • • 
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TAbLE 5 
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' it Test Results' on the Total Test ! 
for the Effect of Or»der of Subtest Administration . 


Order 


N Mean 


S.D. . 


^ SE' 


t 

. Val ue 
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2-Tail 
Probability 


. VC 

■ • , CV 


69 65.923, 
62 62.871 


21.;!99 
'20.354 


2.624 
«v585 
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